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EYE-SCAN BEHAVIOR IN A FLIGHT SIMULATION TASK 


AS A FUNCTION OF LEVEL 01 TRAINING 


Research Objective: 

The present study explored eye-scan behavior as a function 
of level of subject training. The initial phase of the research 
involved the development of a suitable task that was 
representative of that performed by pilots, yet had the necessary 
characteris tics to make research on instrument scanning feasible. 
The second phase of the research involved data collection to 
refine the task and to collect standardization data. The third 
phase consisted of data collection with simultaneous measurement 
of task performance and scanning behavior as initially naive 
subjects practiced the task and developed skill. 


Approach: 

ihe flight simulaton task employed in the study was 
presented on a desk-top microcomputer, and was a simulation of 
some of the tasks performed by aircraft pilots under instrument 
flight conditions. Subjects monitored and controlled the 
heading, altitude, rate of ascent/descent, speed, and attitude 
(pitch and roll) of the simulated aircraft. During specified 
times during each ten-minute trial, instructions were given to 
approach a new heading, a new altitude, or to land the aircraft. 
Task performance data was recorded at intervals of l/10th second. 
Concurrent eye-movement data was collected at intervals of l/30th 
second, by a second microcomputer dedicated to the task of 
oculometer data reduction. The type and time of occurence of 
task instructions and subject control responses were recorded by 
the data reduction microcomputer. 


Progress to Date: 

Preliminary results of the study were reported in a paper 
presented at the Annual Meeting of the Human Factors Society, 
Baltimore Maryland on October 1, 1985. This paper appears in the 
Proceedings of the Annual Meeting of the Human Factors Society, 
and a copy of this paper is attached. 


Future Plans: 

Testing of subjects in increased "workload" situations is 
planned. Such test conditions will permit an examination of 
eye-scan during situations in which tasks must be "shed" in order 
to accomplish the central or most important task. 
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FYE-SCAN BEHAVIOR IN A FLIGHT SIMULATION TASK 
AS A FUNCTION OF LEVEL OF TRAINING 


James R. Comstock, Jr. 

Glynn D. Coates 

Raymond H. Kirby 

Department of Psychology 
Old Dominion University 
Norfolk, Virginia 


ABSTRACT 

The present study explored eye-scan behavior as a function of level of 
subject training. Oculometric (eye-scan) measures were recorded from each of 
ten subjects during training trials on a CRT-based flight simulation task. The 
task developed for the study incorporated subtasks representative of specific 
activities performed by pilots, but which could be performed at asymptotic 
levels within relatively short periods of training. Changes in eye-scan 
behavior were examined as initially untrained subjects developed skill in the 
task. Eye-scan predictors of performance on the task were found. Examination 
of eye-scan in proximity to selected task events revealed differences in the 
distribution of looks at the instruments as a function of level of training. 


INTRODUCTION 

Among the variables that are 
important to an understanding of how 
pilots obtain information from scanning 
aircraft instruments, and to forming a 
basis for improved designs of such 
information display systems, is an 
understanding of what constitutes "good" 
and "poor" scanning behavior on the part 
of the pilot. One approach to 
developing an understanding of what 
constitutes good versus poor scanning 
behavior is to study changes in scanning 
as initially untrained subjects develop 
skill in a flight simulation task. 
Because research to date on aircraft 
instrument scanning has generally 
utilized highly trained pilots with 
highly developed skills and little 
variation in skill from one to another, 
there has been little opportunity to 
examine changes in eye-scan as a 
function of level of training. 

The initial objective of the 
present study was to develop a task that 
employed the types of activities 
required of an operational aircraft 
pilot, but which could be presented less 
expensively and learned more quickly. A 
requirement of the task was that 
asymptotic levels of performance could 
be achieved within relatively short 
periods of training. After developing 
such a task, secondary objectives were 


(1) to study the evolution of eye-scan 
behavior from the initial exposure to 
the task to the eventual asymptotic 
level of performance, and (2) to 
determine if there are eye-scan behavior 
correlates of subject performance on the 
task . 

The conceptual basis of the present 
study is an "error-dependent model of 
instrument scanning behavior" (Jones, 
1983). This model was the product of 
prior research on the instrument 
scanning behavior of commercial airline 
pilots. Briefly, the model assumes that 
the scanning behavior of the pilot is 
driven by the pilot's knowledge of the 
desired state of the aircraft (based on 
experience and training) and the pilot's 
attempt to reduce the discrepancy 
between the actual state of the aircraft 
and the desired state of the aircraft. 
Therefore, instrument scanning repre- 
sents an attempt to obtain information 
that may indicate (1) that the actual 
state of the aircraft does not deviate 
from the desired state enough to warrant 
corrective action, or (2) that the 
actual state deviates from the desired 
state by a certain direction and 
magnitude. 

In the case of a deviation between 
ictual and desired states, pilot control 
responses are initiated which then shift 
eye-scan behavior, according to the 
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modi’ 1 , to a confirmatory loop in which 
eye-scan is driven by the need for 
information confirming that the control 
response was in fact an action that 
would reduc the deviation between 
actual and desired states. It would be 
expected that novice subjects, having 
little experience or training, would 
initially exhibit eye-scan behavior that 
is more information gathering and 
hypothesis testing than would be seen in 
these same subjects after increased 
levels of training and experience. 


METHODOLOGY 
Subjects and Method 

A total of 10 undergraduate college 
students performed a simulated flight 
task by "flying" a total of 25, 
ten-minute flights (trials! over five 
consecutive days. On the initial day, 
each subject received a brief 
standardized description of the purpose 
of the experiment, a description of the 
oculometer, and its purpose, and 
identification of the relatively 
unobtrusive recording equipment adjacent 
to the task computer in the subject's 
room. The subject was then presented 
with a standardized description of the 
task, the flight plan, and detailed 
descriptions of the control mecnanisms 
for the computer based flight 
simulation. A practice trial was then 
presented to the subject (without 
heading or altitude instructions) during 
which the subject had an opportunity to 
manipulate and observe the result of 
each of the simulation controls. After 
the practice trial, and the answering 
any question the subject might have 
regarding the task, the first of the 25 
training trials began. Rest breaks of 
approximately three minutes duration 
were observed at the conclusion of each 
trial. 

The design of the experiment 
permitted within-sub ject assessment of 
performance and eye-scan data. The 
wi thin-subject data permits examination 
of eye-scan during conditions of good 
and poor performance by the same 
individual, enabling changes in eye-scan 
with level of training on the task to be 
eva 1 uated . 

Flight Simulation Task 

The flight simulation task utilized 
in the present study was designed to 
provide subtasks representative of 
specific tasks performed by pilots. The 
f.light simulation task was CRT based. 


and was presented on a desk-top 
micro-computer (Zenith Z-89) , and 
permitted subjects to monitor and 
control the heading, altitude, rate of 
climb/descent, speed, and attitude of 
the simulated aircraft. During 
specified times during each ten-minute 
trial, instructions were given 
(analogous to Air Traffic Control 
communications) to change headings or 
altitude, or to land the aircraft. 
These instructions were issued through 
instruction boxes on the CRT display. 
Tn» instruction box for a particular 
fli^nc parameter, such as heading or 
altitude, was located on the opposite 
side of the CRT display from the 
corresponding indicator, in order (1) to 
reduce the role of peripheral vision 
during performance of the task, and 
(2) to make eye movements between- an 
indicator and its corresponding instruc- 
tion box easy to spot in the oculometric 
data. In addition to the instruction 
boxes, there was a "Warning Box" which 
presented a warning such as "SPEED TOO 
LOW", if a particular flight parameter 
greatly exceeded normal bounds. 

Task performance data on six 
dependent measures were recorded at 
intervals of 0.1 second. These measures 
include (1) Heading change performance, 

(2) Altitude change performance, 

(3) Landing performance, (4) Airspeed 
during heading change, (5) Airspeed 
during altitude change, and (5) Airspeed 
during landing. These six measures were 
available as individual performance 
measures or they could be combined into 
a single composite performance measure. 

Measurement of eye movement 

Eye movement was measured using the 
corneal reflection technique. This 
technique allows unobtrusive measurement 
of eye lookpoint while permitting 
subject head movement over approximately 
one cubic foot of space. Details of the 
technology of the instrument may be 
found in Merchant and Morrisette (1974). 
Specifications of the system used m the 
present study, a NASA Langley Research 
Center modified Honeywell Mark III, may 
be found in Spady (1978). A review of 
various techniques of eye movement 
recording, and typical applications, may 
be found in Young and Sheena (1975). 

Oculometer data was recorded on a 
second micro-computer at a sampling rate 
of 1/30 second (oculometer video frame 
update rate) . The second micro-computer 
was connected to the flignt task 
computer and in addition to oculometer 
data also recorded the time and type of 
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Figure 1: Task Performance, Heading Indicator Dwell Time, and Warning 

Messages as a function of level of training 


each instruction issued to the subject, 
and the time and type of each subject 
control response. This permitted 
subsequent examination of eye-scan 
behavior related to specific task in- 
structions or control responses. 

RESULTS AND DISCUSSION 

Analyses of eye movement data may 
be conducted in various ways, making it 
imperative that terms and measures are 
carefully defined. For the present 
study "dwell time" was defined as the 
time the eyes spend within the 
boundaries of a particular instrument, 
before moving on (saccade) to another 
instrument. 

In addition, eye movement data may 
be analyzed either by examining "global" 
measures such as dwell times on each 
instrument during a particular trial or 
portion of a trial, or by examining the 
data selectively at specific points or 
events in the flight simulation task. 
The results presented here are based on 
both global and selective measures of 
eye-scan behavior. 

Predictors of task performance 

Performance measures presented as a 
function of the number of trials reveals 
that the task is a sensitive measure of 
complex performance that approaches 
asymptotic levels by the 25th trial. 


Figure 1 shows the composite measure of 
task performance over 25 trials, the 
amount of time spent with a task 
"Warning Message", and the average dwell 
time on the Heading Indicator. Heading 
Indicator dwell time is presented in the 
figure as it was the global eye movement 
measure with the highest simple 
correlation with the task composite per- 
formance measure for the combined data 
of all ten subjects. 

Using dwell times on the indicators 
and instruction boxes on the CRT based 
flight simulation task as predictors of 
the task composite performance measure 
in multiple regression analyses, a 
Multiple R of .94 is found for the 
combined data of the ten subjects over 
the 25 trials. If regression equations 
are derived separately for each of the 
subjects, based on one individual's 
dwell times and rate of dwells and his 
or her composite performance index, the 
picture is more complicated, as subjects 
do not share the same set of predictors. 
The failure to find consistent 
predictors using global measures of eye 
scan suggests that other measures be 
explored. These measures include 
examination of sequences of dwells on 
the instruments, and the time required 
for the performance of tlese sequences 
(These measures are currently being 
explored but are not available for the 
present report) . 
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Time-Locked Time-Histories 


GENERAL DISCUSSION 


Figure 2 presents a new way of 
looking at eye-scan information. These 
plots are labelled time-locked 
time-histories as they illustrate where 
the subject was looking during a period 
of time before and after a certain 
time-locking event. The time-locking 
event for the time histories shown in 
figure 2 was the presentation of a new 
heading instruction. Time “0" is when 
this instruction was issued, and time 
”1" represents 1 second after the event. 
The gray bands on the vertical scale 
represent percentage of time spent 
looking at each instrument over a number 
of occurrences of the time-locking 
event. The total vertical heighth at a 
given point corresponds to 100 percent. 
A wide gray band would mean that a high 
percentage of time was spent looking at 
a particular point during the time 
locking event interval, a narrow band 
indicates a low percentage of looks at a 
particular place at a particular point 
in time. Since there were only five new 
heading instructions per trial, each of 
the time histories shown in figure 2 was 
based on combining five trials for a 
total of 25 occurrences of the 
time-locking event. 

The panel of figures 2a through 2d 
show the time-locked time-histories for 
one subject for increased levels of 
training. As shown in figure 2a, in 
response to a new heading instruction 
this subject showed an increase in looks 
at the heading instruction box, followed 
by an increase in looks at the heading 
indicator. Also in figure 2a, the 
continued presence of dwells on the 
heading indicator probably reflect 
multiple looks at this display to insure 
that it is changing in the desired 
direction. Continued looks at the 
heading indicator diminish with 
increased training, as can be seen by 
comparing figures 2a through 2d. 

Perhaps the most noticeable change 
with training level is the' increase in 
looks at the Pitch/Roll indicator 
following a heading change instruction. 
The subject, with experience, learns 
that a change in heading requires a 
change in both pitch and roll. Methods 
of quantifying the changes in eye-scan 
behavior during time-locked events are 
currently being explored. 


The results of the present study 
show changes in eye-scan behavior as a 
function of level of training. These 
changes are revealed in both global and 
specific measures of eye-scan. Average 
dwell time, for example, showed a 
consistent increase with increased 
tra.ning. Examination of eye-scan in 
proximity to selected task events 
revealed differences in the distribution 
of looks at the instruments as a 
function of level of training (this is 
shown in Figure 2). 

The capability of identifying 
eye-scan behavior as varying in quality 
has important potential for establishing 
criteria against which to evaluate 
different information displays, for 
improving pilot training programs, and 
may offer some possibility for improved 
selection of pilot training candidates. 
The development and utilization of new 
task-sensitive eye-scan measures may 
permit addressing these problems. 
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ASSESSMENT OF SPECIFIC INFORMATION NEEDS COMPLETED 


Research Objective: 

The initial experiment concerned the assessment of transfer 
of visual information from cockpit displays to the flight crew as 
measured by oculometric techniques. Given that the time spent 
viewing a display may vary depending on the nature of the 
information sought by the pilot examining the display, this 
experiment permitted an evaluation of time on the display in 
response to varied tasks requiring the pilot (1) to confirm 
indicator or needle position, (2) to determine if the indicated 
value is changing or not, or (3) to determine change of rate of 
the indicated value. The design of the function of new displays 
and display technologies can be facilitated only through an 
understanding of the nature of the information presented in these 
displays . 


Approach: 

The initial experiment was conducted using the ATC-510 
desk-top fixed-base simulator located at the NASA Langley 
Research Center. In addition to the primary flight instruments 
employed in the simulation, a secondary task was employed using 
an additional electromechanical display situated along the right 
side of the simulator panel. For the test subjects, the task was 
to fly the simulator on a heading of 60 degrees and at an 
altitude of 3160 feet, while simultaneously performing the 
secondary task. The secondary task involved extracting selected 
information from the secondary task display, and responding by a 
pushbutton located on the simulator "wheel". 


Progress to Date: 

Data collection for the initial study has been completed. A 
report on the findings of the study is in preparation. 


STUDY OF EYE -DIRECTED COMPUTER INPUT TECHNIQUES COMPLETED 


Dr* J. Raymond Comstock, Jr. Dr. Randall L. Harris, Sr.' 

Old Dominion Uni varsity Crew/Vehicle Intarfaca Rasaarch Branch 

Talaphona: (804) 440-4439 1 Talaphona: (804) 863-3917 

Saptambar 13, 1983 
RTOP 505-35-33 


Rasaarch Objactlvas 


Although tha Intarfaca between tha pilot and his vahlcla Is bacomlng mora comp I ax with 
aach naw alrplana daslgn. It Is not Inconcalvabla to think that ona approach to Intarfaca 
simplification would put tha pilot of tomorrow at a dask with nothing but Input/output 
(I/O) davlcas, flight control lars and a larga graphic display llnkad to tha aircraft 
computar systam. Computar manufacturars ara constantly trying naw tachnlquas to Improva 
tha communications Intarfaca batwaan tha oparator and tha computar. Such I/O candldatas 
as kayboards. Joysticks, trackballs, votca, and mousas ara balng Implemented and studied. 
Many of these candldatas may be applicable to tha flight environment, however, with tha 
exception of voice, they aach have tha disadvantage that hand movement Is required. Ona 
technique of Input which leaves tha hands free to do other types of tasks Is currently 
balng explored by LaRC. This technique Is tha use of eye-directed computar Input (display 
menu selection) and It Is balng studied under rasaarch grant at tha Old Dominion 
University. 


In order to compare eye-directed Input with a "bast case" (hand directly on tha keypad) 
keyboard entry systam, 4-cholce and 12-cholce visual search tasks ware designed and 
evaluated. Langley's oculometer systam was used to track subjects lookpolnts during a 
menu selection, task. A microcomputer was used to present to tha subject a menu and tha 
Item to be selected by eye position. Tha accompanying figure shows tha format of tha 
CRT-based 12-cholce search task. Tha dashed lines within box 4 on tha figure show where 
tha subject Is looking. These lines ara analogous to the traditional "cursor", except 
that this cursor Is eye-directed and raqulras no hand movement by the subject to move It 
from place to place. Tha selection of an Item from a menu is therefore reduced to the 
pressing of a single button and could be Implemented, at a future data, entirely by eye 
movement. 


Accomplishment Description 


Tha results of tha preliminary study showed that the use of eye-directed inputs made the 
selection process significantly faster In both 4- and 12-cholce visual search tasks. The 
faster response times with eye-directed Input were found when comparing "best case" 
keyboard Inputs (hand directly on keypad) with eye-directed Input. The time saved by 
eye-directed Input was shown to be greater when the menu Incorporates a greater number of 
Items. The use of eye-directed Inputs Is hypothesized to have a psychological advantage 
over other Input devices In that It provides the user with a sense of enhanced control 
over the machine, because the machine appears to respond Immediately to the user's 
Intentions. 


Future Plans 

It Is planned that future research will extend the eye-directed Input technique to permit 
100 percent eye-directed Input (no push-button required for selection entry) and that the 
eye-directed technique will be compared with other Input techniques such as voice, 
trackball, or mouse. These studies will Investigate both speed and accuracy cf responses. 
Further, plans are being made to move the eye-directed Input technique Into a flight 
simulator environment to determine Its efficacy for selected operational tasks. 
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